Lesions to different regions of frontal cortex have dissociable effects on voluntary persistence.

Deciding how long to keep waiting for uncertain future rewards is a complex problem. Previous research has shown that choosing to stop waiting results from an evaluative process that weighs the subjective value of the awaited reward against the opportunity cost of waiting. In functional neuroimaging data, activity in ventromedial prefrontal cortex (vmPFC) tracks the dynamics of this evaluation, while activation in the dorsomedial prefrontal cortex (dmPFC) and anterior insula (AI) ramps up before a decision to quit is made. Here, we provide causal evidence of the necessity of these brain regions for successful performance in a willingness-to-wait task. 28 participants with frontal lobe lesions were tested on their ability to adaptively calibrate how long they waited for monetary rewards. We grouped the participants based on the location of their lesions, which were primarily in ventromedial, dorsomedial, or lateral parts of their prefrontal cortex (vmPFC, dmPFC, and lPFC, respectively), or in the anterior insula. We compared the performance of each subset of lesion participants to behavior in a control group without lesions (n=18). Finally, we fit a newly developed computational model to the data to glean a more mechanistic understanding of how lesions affect the cognitive processes underlying choice. We found that participants with lesions to the vmPFC waited less overall, while participants with lesions to the dmPFC and anterior insula were specifically impaired at calibrating their level of persistence to the environment. These behavioral effects were accounted for by systematic differences in parameter estimates from a computational model of task performance: while the vmPFC group showed reduced initial willingness to wait, lesions to the dmPFC/anterior insula were associated with slower learning from negative feedback. These findings corroborate the notion that failures of persistence can be driven by sophisticated cost-benefit analyses rather than lapses in self-control. They also support the functional specialization of different parts of the prefrontal cortex in service of voluntary persistence.

Reinforcement-based responsiveness, depression, and anhedonia: A multi-method investigation of intergenerational risk.

Offspring of depressed parents are at an increased risk for depression. Reward- and punishment-based systems might be mechanisms linking maternal outcomes to offspring depression and anhedonia. The current study was designed to investigate the intergenerational relations between maternal markers of reward and punishment responsiveness and their offspring's depression and anhedonia in a community sample of 40 mother (mean age = 44.5; SD = 6.82) and adolescent (mean age = 14.73; SD = 1.25; 52.5% female) dyads. Maternal markers of reward and punishment responsiveness were captured using self-report, behavioral, and neurophysiological methods, and self-reported depression and anhedonia symptoms were used as outcomes among the adolescent offspring. Maternal self-reported reward responsiveness and punishment learning rates were differentially associated with depression across male and female offspring. Regarding anhedonia, maternal punishment learning rate was positively related to adolescent anhedonia regardless of offspring biological sex. Maternal reward learning rate was also positively associated with anhedonia among male offspring. In general, low concurrence across self-report, behavioral, and neurophysiological markers of reward and punishment responsiveness was found. The results from the current study suggest that learning-rates on reinforcement-based behavioral tasks may be important objective markers to consider when evaluating intergenerational risk.

Leveraging Decision Science to Characterize Depression.

This brief review examines the potential to use decision science to objectively characterize depression. We provide a brief overview of the existing literature examining different domains of decision-making in depression. Because this overview highlights the specific role of reinforcement learning as an important decision process affected in the disorder, we then introduce reinforcement learning modeling and explain how this approach has identified specific reinforcement learning deficits in depression. We conclude with ideas for future research at the intersection of decision science and depression, emphasizing the potential for decision science to help uncover underlying mechanisms and targets for the treatment of depression.

Reinforcement learning with associative or discriminative generalization across states and actions: fMRI at 3 T and 7 T.

The model-free algorithms of "reinforcement learning" (RL) have gained clout across disciplines, but so too have model-based alternatives. The present study emphasizes other dimensions of this model space in consideration of associative or discriminative generalization across states and actions. This "generalized reinforcement learning" (GRL) model, a frugal extension of RL, parsimoniously retains the single reward-prediction error (RPE), but the scope of learning goes beyond the experienced state and action. Instead, the generalized RPE is efficiently relayed for bidirectional counterfactual updating of value estimates for other representations. Aided by structural information but as an implicit rather than explicit cognitive map, GRL provided the most precise account of human behavior and individual differences in a reversal-learning task with hierarchical structure that encouraged inverse generalization across both states and actions. Reflecting inference that could be true, false (i.e., overgeneralization), or absent (i.e., undergeneralization), state generalization distinguished those who learned well more so than action generalization. With high-resolution high-field fMRI targeting the dopaminergic midbrain, the GRL model's RPE signals (alongside value and decision signals) were localized within not only the striatum but also the substantia nigra and the ventral tegmental area, including specific effects of generalization that also extend to the hippocampus. Factoring in generalization as a multidimensional process in value-based learning, these findings shed light on complexities that, while challenging classic RL, can still be resolved within the bounds of its core computations.

An energizing role for motivation in information-seeking during the early phase of the COVID-19 pandemic.

The COVID-19 pandemic has highlighted the importance of understanding and managing information seeking behavior. Information-seeking in humans is often viewed as irrational rather than utility maximizing. Here, we hypothesized that this apparent disconnect between utility and information-seeking is due to a latent third variable, motivation. We quantified information-seeking, learning, and COVID-19-related concern (which we used as a proxy for motivation regarding COVID-19 and the changes in circumstance it caused) in a US-based sample (n = 5376) during spring 2020. We found that self-reported levels of COVID-19 concern were associated with directed seeking of COVID-19-related content and better memory for such information. Interestingly, this specific motivational state was also associated with a general enhancement of information-seeking for content unrelated to COVID-19. These effects were associated with commensurate changes to utility expectations and were dissociable from the influence of non-specific anxiety. Thus, motivation both directs and energizes epistemic behavior, linking together utility and curiosity.

Learned temporal statistics guide information seeking and shape memory.

Curiosity drives information seeking and promotes learning. Prior work has focused on how curiosity is elicited by intrinsic qualities of information, leaving open questions about how curiosity, exploration, and learning are shaped by the environment. Here we examine how temporal dynamics of the learning environment shape curiosity and learning. Participants (n = 71) foraged for the answer to trivia questions in two conditions that differed only in their temporal statistics. In one condition, the timing of information delivery followed a uniform distribution, while in another it followed a heavy-tailed distribution. We found that the two conditions elicited distinct responses in both behavior and pupil dilation: participants were more likely to wait for information and to later remember it in the uniform distribution. By contrast, participants showed greater surprise, evidenced in a spike in pupil dilation, when presented with the answers in the heavy-tailed distribution. Furthermore, pupil dilation was inversely related to curiosity and memory, suggesting that temporal uncertainty may interfere with the positive effects of curiosity on learning. Our findings demonstrate that the predicted timing of information delivery influences information seeking, memory, and physiological arousal, suggesting that information is best learned when it is both intrinsically interesting and presented within a temporally predictable environment. (PsycInfo Database Record (c) 2022 APA, all rights reserved).